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ABSTRACT 

This report summarizes short-term impacts of the Quantum 
Opportunity Program (QOP) demonstration, which helped at-risk high school-age 
youth graduate and enroll in postsecondary education or training. QOP 
involved intensive case management and mentoring emphasizing after school 
supplemental academic education, developmental activities, and community 
service. It served a single cohort of youth from ninth grade in 1995 through 
fall 2000. A local community based organization in each of six inner city 
communities and one rural community teamed with a school to implement and 
operate a QOP program. The study examined how well the program was 
implemented at each site, how much it cost, how much time enrollees spent on 
program activities, and how QOP affected enrollees in the short term. Data 
from site visits, QOP conferences, conference calls with QOP staff, and 
student surveys indicated that QOP achieved short-term success in raising 
high school completion rates and enrollment in post secondary education or 
other training. It did not improve grades and achievement test scores or 
reduce risky behaviors . QOP benefited enrollees in the middle of the eligible 
grade distribution more than enrollees at the top or bottom. Most enrollees 
attended very few program activities. Impact varied by site. Impacts for the 
whole QOP demonstration were substantially attributable to the impacts of two 
sites. (SM) 
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Executive Summary 



From July 1995 through September 2001, the U.S. Department of Labor (DOL) and The Ford 
Foundation (Ford) operated a demonstration of the Quantum Opportunity Program (QOP). QOP 
offered intensive and comprehensive services to help at-risk youth graduate from high school and 
enroll in postsecondary education or training. The QOP demonstration included several features of 
Workforce Investment Act (WIA) youth programs, and findings from the demonstration might 
provide some insight about the implementation challenges that such WIA programs will encounter 
and the potential effectiveness of those programs. 

The QOP demonstration targeted youth with low grades entering high schools with high 
dropout rates. Randomly selected eligible youth were enrolled in QOP and served even if they 
transferred to other schools, dropped out of school, became incarcerated, or became inactive in 
QOP for a long time. QOP’s primary goals were to increase the rates of high school graduation and 
enrollment in postsecondary education or training. Its secondary goals were to improve high school 
grades and achievement test scores and to reduce risky behaviors, such as substance abuse, crime, 
and teen parenting. 

QOP was mainly an after-school program providing case management and mentoring, 
supplemental education, developmental activities, community service activities, supportive services, 
and financial incentives. These services were provided year-round for five years to enrollees who 
had not graduated from high school, and were designed to be comprehensive enough to address all 
barriers to success and to be intensive. The program model specified roughly 15 to 25 enrollees per 
case manager, and it prescribed an annual participation goal of 750 hours for each enrollee who had 
not graduated. From graduation to the end of the demonstration, enrollees who had graduated 
received limited services — some mentoring and assistance with enrolling in postsecondary education 
or training. 

Community-based organizations (CBOs) in seven sites operated QOP demonstration programs. 
Five sites (Cleveland, Fort Worth, Plouston, Memphis, and Washington, D.C.) were funded by 
DOL. Four of the five served 100 youth each, and the Washington, D.C., site served 80 youth. The 
other two sites (Philadelphia and Yakima) served 50 youth each with funding from Ford. DOL has 
also funded an evaluation of the QOP demonstration. 

Evaluation Design 

Through annual site visits, annual QOP conferences, and conference calls with QOP staff, we 
assessed how well the CBOs in the QOP demonstration implemented the program model. From 
information provided by QOP staff, we also measured how much QOP cost and how much 
enrollees participated in QOP. To estimate QOP’s impacts on high school performance and 
graduation, postsecondary education or training, and risky behaviors, we conducted two surveys, 
administered achievement tests in reading and mathematics, and collected high school transcripts for 
a group of youth who were enrolled in QOP and a group of statistically identical youth — the control 
group — who were not allowed to participate in QOP. We formed the QOP and control groups at 
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the start of the demonstration by randomly assigning each of the nearly 1,100 youth eligible for the 
program to one group or the other. 

How Well Was QOP Implemented? 

Every site implemented a version of QOP. However, two sites implemented a version of QOP 
that deviated substantially from the program model, and the other five sites implemented versions 
that deviated moderately from the model. With the exception of the Philadelphia site where the 
program was operated by the CBO that helped to design the QOP model, local CBOs found 
implementing QOP difficult, primarily because QOP was substantially more comprehensive, 
intensive, and complex than their traditional programs. Other implementation findings include the 
following: 

• Most sites implemented the mentoring component as prescribed by the QOP model. 
Case managers developed deep personal relationships with the 40 to 60 percent of 
enrollees who attended some program activities regularly and addressed a wide range of 
barriers facing those youth. QOP’s policy of providing access to services regardless of 
the enrollee’s behaviors (that is, becoming incarcerated, moving to another community, 
dropping out of high school) was well implemented. 

• Few sites implemented the education component as prescribed. In most sites, tutors 
with education backgrounds were not provided on a long-term, consistent basis. Few 
DOL-funded sites implemented computer-assisted instruction as prescribed. While 
several sites assessed the educational achievement of enrollees, few sites translated those 
assessments into meaningful individualized education plans. 

• All sites implemented the developmental component, although with a greater emphasis 
on recreational activities than suggested by the QOP model. 

• No site implemented the community service component as prescribed. Sites scheduled 
only a small fraction of the prescribed number of community service activities. 

• All sites implemented stipends successfully. All sites implemented accrual accounts, 
although DOL-funded sites did not provide regular account statements to enrollees. 
Several sites provided bonuses to enrollees who completed major program activities. 

Most sites provided adequate food and, eventually, transportation services, but few sites 
provided adequate child care services or screening and referral for the physical and mental health 
needs of enrollees. 

How Much Did Enrollees Participate? 

Most enrollees attended relatively few program activities. Enrollees spent an average of 174 
hours per year on QOP activities — 23 percent of the annual goal of 750 hours — through the first 
four years of the demonstration. The average fell steadily from 247 hours in the first year to 89 
hours in the fourth year, while the fraction of enrollees spending no time at all on QOP activities 
rose steadily from 1 percent to 36 percent. 
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Participation varied substantially from site to site. The average annual participation in the two 
Ford-funded sites was 294 hours — more than twice the average annual participation of 126 hours in 
the five DOL-funded sites. 

How Much Did QOP Cost? 

The total cost per enrollee over the full five-year demonstration period was $18,000 to $22,000 
for DOL-funded sites, $23,000 for the Yakima site, and $49,000 for the Philadelphia site. 

What Were QOP’s Short-Term Impacts? 

Primary Outcomes 

• QOP increased the likelihood of graduation. QOP increased by a statistically 
significant seven percentage points the likelihood that enrollees graduated from high 
school with a diploma. 

• QOP increased the likelihood of engaging in postsecondary education or 
training. The size and statistical significance of the impact, however, depends on how 
this outcome was measured and how the impact was estimated. 

Secondary Outcomes 

• QOP did not improve grades or achievement test scores. 

• QOP did not reduce risky behaviors. QOP did not significantly reduce any risky 
behavior, including gang activity, crime, and teen parenting, and according to data from 
one of the evaluation surveys, QOP significantly increased the fraction of enrollees who 
had a drink and the fraction who used an illegal drug in the 30 days before the survey. 
However, some evidence suggests that there were differences between QOP enrollees 
and control-group youth in the accuracy with which they reported risky behaviors. 
Those differences might have contributed substantially to the estimated detrimental 
impacts on drinking and drug use. That QOP might not have increased drinking and 
drug use is also suggested by data from a second evaluation survey. According to those 
data, QOP had beneficial — but not significant — impacts on drinking and drug use. 

Subgroup and Site Impacts 

• QOP helped some enrollees more than others. For example, QOP was more 
beneficial for enrollees in the middle of the eligible grade distribution than for enrollees 
at the top or bottom of the distribution. For enrollees in the middle of the distribution, 
QOP significantly increased the likelihood of graduating from high school, the 
likelihood of graduating or earning a GED, and the likelihood of attending or being 
accepted by a college. QOP significantly reduced for these enrollees the likelihood of 
having a child. 
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• QOP’s impacts varied from site to site. And, only one of the seven sites — the 
Cleveland site — had significant beneficial impacts and no significant detrimental 
impacts. The Cleveland site significandy increased the likelihood of graduating from 
high school, significandy increased the likelihood of attending or being accepted by a 
college, and significandy decreased the likelihood of binge drinking. 

• Impacts for the whole demonstration were substantially — but not entirely — 
attributable to the impacts of the Philadelphia site alone or the Philadelphia and 
Yakima sites, the Ford-funded sites, together. The Ford-funded sites significandy 
increased mathematics achievement and the likelihood of engaging in postsecondary 
education or training, significandy increased the combined likelihood of engaging in 
postsecondary education or training or having a good job, and significandy reduced the 
likelihood of having a child. However, these sites also significandy increased the 
likelihood of binge drinking, using an illegal drug, and committing a crime, although 
these detrimental impacts might not have been attributable to QOP for the reasons 
noted above. The DOL-funded sites significandy increased the likelihood of graduating 
from high school, one of QOP’s primary objectives. QOP had no other significant 
impacts in the DOL-funded sites. 

These impacts are short-term impacts because we estimated them from data collected during 
the fourth and fifth years of the demonstration, that is, before the demonstration was over and when 
many youth were either still attending high school or had only recendy graduated. Longer-term 
impacts, which may be a more appropriate basis for policy decisions, might be more or less 
beneficial than the short-term impacts presented in this report. To measure longer-term impacts, 
DOL is having us collect data in fall 2002 and fall 2004. The fall 2002 data collection is roughly 
seven years after the youth in the demonstration sample entered the ninth grade and two years after 
the end of the QOP demonstration. 
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Introduction 



This report summarizes the short-term impacts of the Quantum Opportunity Program (QOP) 1 
demonstration. From July 1995 through September 2001, the U.S. Department of Labor (DOL) 
and The Ford Foundation (Ford) operated a demonstration of QOP designed to help at-risk 2 high- 
school-age youth graduate from high school and enroll in postsecondary education or training. 
QOP was an intensive case management and mentoring program that emphasized after-school 
supplemental academic education, developmental activities, and community service. The QOP 
demonstration included several features of Workforce Investment Act (WIA) youth programs, and 
findings from the demonstration might provide some insight about the implementation challenges 
that such WIA programs will encounter and the potential effectiveness of those programs. 

QOP is one of several approaches to assisting at-risk youth evaluated in recent years by DOL 
and the Department of Education (ED), including Job Corps, Job Training Partnership Act (JTPA) 
youth programs, Career Academies, Center for Employment Training (CET), Upward Bound, and 
Talent Search. As employers demanded more advanced technical skills and work-readiness skills in 
entry-level employees, DOL and ED became concerned that some youth are not effectively 
prepared to meet these rising standards. Such youth are at increased risk of unemployment, welfare 
dependency, substance abuse, criminal activity, and teenage childbearing. Finding effective 
approaches to assisting these youth in achieving economic self-sufficiency is critical to avoiding the 
personal losses resulting from such life events and to reducing the costs of providing assistance 
through Unemployment Insurance, WIA, Temporary Assistance for Needy Families (TANF), 
Medicaid, and other public programs. 

The motivation for DOL and Ford to conduct the QOP demonstration arose from QOP’s 
history. In the late 1980s, three organizations — Opportunities Industrialization Centers of America 
(OICA) in Philadelphia; Ford; and Remediation and Training Institute in Alexandria, Virginia — 
developed the QOP model. From 1989 through 1993, Ford funded and OICA operated a small- 
scale pilot of QOP in six sites. The Center for Human Resources at Brandeis University evaluated 
the pilot, obtaining some findings that were encouraging to DOL and Ford. In early 1995, DOL 
and Ford agreed to test QOP on a larger scale via a demonstration with two sites under private 
management and administration and five sites under federal management and administration, 
specifically, under the demonstration title of JTPA. 

The QOP demonstration served a single cohort of youth from the beginning of the ninth grade 
in the fall of 1995 through the fall of 2000. 3 A local community-based organization (CBO) in each 
of six inner-city communities and one rural community implemented and operated a QOP program. 
Each CBO teamed with from one to three high schools and had 50, 80, or 100 youth enrolled in the 
program. By the end of the demonstration, enrollees were in a variety of statuses, including 

1 The acronym QOP is customarily pronounced kwap. 

2 At-risk youth are at a greater risk of substance abuse, criminal activity, teenage childbearing, not completing high 
school, or not enrolling in a postsecondary education or training program, compared to the average high- school-age 
youth in the United States. 

3 All events occurred one year later in the Washington, D.C., site. 
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attending college or another postsecondary training program, still attending high school, attending a 
general educational development (GED) certification program, working after finishing high school, 
and working or unemployed after dropping out of high school. 

The primary objectives of the demonstration were to increase the likelihood of high school 
completion and to increase enrollment in postsecondary education or training. Its secondary 
objectives were to increase academic achievement while in high school and to reduce risky 
behaviors, such as substance abuse, crime, and teenage childbearing. Under contract to DOL, 
Mathematica Policy Research is evaluating the QOP demonstration, and has assessed the program’s 
implementation and short-term impacts. Detailed findings obtained to date are presented in Schirm 
et al. (2003) and Maxfield et al. (2003). The program impacts reported herein are short-term in that 
they are based on data collected during the fourth and fifth years of the demonstration, that is, while 
sites were still providing services to enrollees and when many youth were either still attending high 
school or had only recently graduated. Future reports will present longer-term program impacts. 

This report summarizes the findings obtained to date and answers the following questions: 

• How well was the QOP program model implemented in the demonstration sites? 

• How much did QOP cost? 

• How much time did enrollees spend on program activities? 

• How did QOP affect enrollees in the short run? 

The report also seeks to determine whether the implementation, cost, and participation findings 
suggest why QOP had the short-term impacts that it did. Before summarizing the evaluation 
findings, we briefly describe the QOP target group and program model in the next section. 



The QOP Target Group and Program Model 



The target group in the QOP demonstration was youth entering the ninth grade in fall 1995 
who met the following criteria: 

• Began the ninth grade at a high school selected for the QOP demonstration. Each high 
school had a dropout rate of 40 percent or more. 

• Were not repeating the ninth grade. 

• Were not so physically disabled or learning disabled that participation in the program 
would not be appropriate, as determined by the school. 

• Had a grade point average (GPA) below the 67 th percentile among the students meeting 
the first three requirements. (The GPA was calculated from grades received in the 
eighth grade.) 



